Simple Reliable Multicast for Parallel Processing in Extended LANs

A typical problem for a parallel processing system involves broadcasting large amounts of data from a master to several worker programs. This paper describes a reliablemulticast method to reduce the communication costs of this distribution. Our solution relies on a tuple-space mechanism as implemented in the Synergy system. We present results showing that even a simple implementation of reliable multicast can dramatically improve performance. 1. Background: The Synergy system The tuple space model was introduced in the LINDA [2] system. Synergy [4] is a tuple-space based SIMD parallel processing system that runs on a network of workstations. From a programmer’s point of view, Synergy consists of three types of entities: exactly one master, one or more workers, and one or more tuple spaces. The master and workers are user programs, while tuple spaces are object repositories to/from which the master and workers can write/read objects called tuples. A tuple is a named data object. Once put into a tuple space, tuples are persistent until removed. Another example of a tuple-space based system is LIMBO [1] which uses reliable multicast in conjunction with tuple spaces to provide Quality of Service management in mobile-aware distributed applications. A parallel solution to a problem is implemented in Synergy by having the master divide the problem space into sub-problems, and then put one or more tuples for each subproblem into tuple space. The workers then obtain tuples from the tuple space, compute the (partial) solution and return the solution to the same or another tuple space. The master, after reading all the solution tuples, generates the complete solution and returns the result. Currently, workers use Synergy system calls to read tuples from the tuple space over a network. Prior to this work, Synergy system calls used only unicast communication. 2. Multicasting In parallel computing, the total solution cost is the sum of the computation and communication costs. For many problems the total cost is dominated by the initial problem distribution cost, i.e. the time to distribute the sub-problem and relevant data to the workers. In cases where a large proportion of the sub-problem data to be distributed is identical, multicasting can improve performance. IP Multicasting is inherently unreliable. In unicasting, reliability is acheived by having the sender retransmit. However, unicast reliability techniques are not scalable for multicast due to the well-known ack-implosion problem. Many sophisticated reliable multicast algorithms have been investigated [3]. In this paper, we show that significant gains can be obtained even with a minimal use of multicast. Specifically we have exploited two features of Synergy: (1) All data to be distributed is present in the tuple space and the tuple space is persistent. (2) Workers normally go to the tuple space to get their sub-problems. We shift the responsibility of reliability from the sender to the receiver. The idea can be summarized as: MASTER: (1) Generate the data and place in tuple space. (2) Wait for acks from workers indicating readiness. (3) Multicast the data. WORKER: (1) Send ack to master indicating worker is ready. (2) Receive data via multicast. (3) If the next expected tuple is not received within an (applicationspecific) interval then retrieve the lost data from tuple space using regular Synergy functions. 3. Experiment Design and Analysis of Results The experiment involved distributing an integer matrix to processors, as in a parallel implementation of matrix multiplication. We calculated the average delay over ten repetitions of the experiment for each combination of